Careful at Estimation and Bold at Exploration

Exploration strategies in continuous action space are often heuristic due to the infinite actions, and these kinds of methods cannot derive a general conclusion. In prior work, it has been shown that policy-based exploration is beneficial for continuous action space in deterministic policy reinforcement learning(DPRL). However, policy-based exploration in DPRL has two prominent issues: aimless exploration and policy divergence, and the policy gradient for exploration is only sometimes helpful due to inaccurate estimation. Based on the double-Q function framework, we introduce a novel exploration strategy to mitigate these issues, separate from the policy gradient. We first propose the greedy Q softmax update schema for Q value update. The expected Q value is derived by weighted summing the conservative Q value over actions, and the weight is the corresponding greedy Q value. Greedy Q takes the maximum value of the two Q functions, and conservative Q takes the minimum value of the two different Q functions. For practicality, this theoretical basis is then extended to allow us to combine action exploration with the Q value update, except for the premise that we have a surrogate policy that behaves like this exploration policy. In practice, we construct such an exploration policy with a few sampled actions, and to meet the premise, we learn such a surrogate policy by minimizing the KL divergence between the target policy and the exploration policy constructed by the conservative Q. We evaluate our method on the Mujoco benchmark and demonstrate superior performance compared to previous state-of-the-art methods across various environments, particularly in the most complex Humanoid environment.Comment: 20 page

SSBM: A Signed Stochastic Block Model for Multiple Structure Discovery in Large-Scale Exploratory Signed Networks

Signed network structure discovery has received extensive attention and has become a research focus in the field of network science. However, most of the existing studies are focused on the networks with a single structure, e.g., community or bipartite, while ignoring multiple structures, e.g., the coexistence of community and bipartite structures. Furthermore, existing studies were faced with challenge regarding large-scale signed networks due to their high time complexity, especially when determining the number of clusters in the observed network without any prior knowledge. In view of this, we propose a mathematically principled method for signed network multiple structure discovery named the Signed Stochastic Block Model (SSBM). The SSBM can capture the multiple structures contained in signed networks, e.g., community, bipartite, and coexistence of them, by adopting a probabilistic model. Moreover, by integrating the minimum message length (MML) criterion and component-wise EM (CEM) algorithm, a scalable learning algorithm that has the ability of model selection is proposed to handle large-scale signed networks. By comparing state-of-the-art methods on synthetic and real-world signed networks, extensive experimental results demonstrate the effectiveness and efficiency of SSBM in discovering large-scale exploratory signed networks with multiple structures

Learning the Network of Graphs for Graph Neural Networks

Graph neural networks (GNNs) have achieved great success in many scenarios with graph-structured data. However, in many real applications, there are three issues when applying GNNs: graphs are unknown, nodes have noisy features, and graphs contain noisy connections. Aiming at solving these problems, we propose a new graph neural network named as GL-GNN. Our model includes multiple sub-modules, each sub-module selects important data features and learn the corresponding key relation graph of data samples when graphs are unknown. GL-GNN further obtains the network of graphs by learning the network of sub-modules. The learned graphs are further fused using an aggregation method over the network of graphs. Our model solves the first issue by simultaneously learning multiple relation graphs of data samples as well as a relation network of graphs, and solves the second and the third issue by selecting important data features as well as important data sample relations. We compare our method with 14 baseline methods on seven datasets when the graph is unknown and 11 baseline methods on two datasets when the graph is known. The results show that our method achieves better accuracies than the baseline methods and is capable of selecting important features and graph edges from the dataset. Our code will be publicly available at \url{https://github.com/Looomo/GL-GNN}

Multi-Modality Multi-Scale Cardiovascular Disease Subtypes Classification Using Raman Image and Medical History

Raman spectroscopy (RS) has been widely used for disease diagnosis, e.g., cardiovascular disease (CVD), owing to its efficiency and component-specific testing capabilities. A series of popular deep learning methods have recently been introduced to learn nuance features from RS for binary classifications and achieved outstanding performance than conventional machine learning methods. However, these existing deep learning methods still confront some challenges in classifying subtypes of CVD. For example, the nuance between subtypes is quite hard to capture and represent by intelligent models due to the chillingly similar shape of RS sequences. Moreover, medical history information is an essential resource for distinguishing subtypes, but they are underutilized. In light of this, we propose a multi-modality multi-scale model called M3S, which is a novel deep learning method with two core modules to address these issues. First, we convert RS data to various resolution images by the Gramian angular field (GAF) to enlarge nuance, and a two-branch structure is leveraged to get embeddings for distinction in the multi-scale feature extraction module. Second, a probability matrix and a weight matrix are used to enhance the classification capacity by combining the RS and medical history data in the multi-modality data fusion module. We perform extensive evaluations of M3S and found its outstanding performance on our in-house dataset, with accuracy, precision, recall, specificity, and F1 score of 0.9330, 0.9379, 0.9291, 0.9752, and 0.9334, respectively. These results demonstrate that the M3S has high performance and robustness compared with popular methods in diagnosing CVD subtypes

Data and Knowledge Co-driving for Cancer Subtype Classification on Multi-Scale Histopathological Slides

Artificial intelligence-enabled histopathological data analysis has become a valuable assistant to the pathologist. However, existing models lack representation and inference abilities compared with those of pathologists, especially in cancer subtype diagnosis, which is unconvincing in clinical practice. For instance, pathologists typically observe the lesions of a slide from global to local, and then can give a diagnosis based on their knowledge and experience. In this paper, we propose a Data and Knowledge Co-driving (D&K) model to replicate the process of cancer subtype classification on a histopathological slide like a pathologist. Specifically, in the data-driven module, the bagging mechanism in ensemble learning is leveraged to integrate the histological features from various bags extracted by the embedding representation unit. Furthermore, a knowledge-driven module is established based on the Gestalt principle in psychology to build the three-dimensional (3D) expert knowledge space and map histological features into this space for metric. Then, the diagnosis can be made according to the Euclidean distance between them. Extensive experimental results on both public and in-house datasets demonstrate that the D&K model has a high performance and credible results compared with the state-of-the-art methods for diagnosing histopathological subtypes. Code: https://github.com/Dennis-YB/Data-and-Knowledge-Co-driving-for-Cancer-Subtypes-Classificatio

Pressure-tunable magnetic topological phases in magnetic topological insulator MnSb4Te7

Magnetic topological insulators, possessing both magnetic order and topological electronic structure, provides an excellent platform to research unusual physical properties. Here, we report a high-pressure study on the anomalous Hall effect of magnetic TI MnSb4Te7 through transports measurements combined with first-principle theoretical calculations. We discover that the ground state of MnSb4Te7 experiences a magnetic phase transition from the A-type antiferromagnetic state to ferromagnetic dominating state at 3.78 GPa, although its crystal sustains a rhombohedral phase under high pressures up to 8 GPa. The anomalous Hall conductance {\sigma}xyA keeps around 10 {\Omega}-1 cm-1, dominated by the intrinsic mechanism even after the magnetic phase transition. The results shed light on the intriguing magnetism in MnSb4Te7 and pave the way for further studies of the relationship between topology and magnetism in topological materials.Comment: 10 pages, 4 figure

IASM: A System for the Intelligent Active Surveillance of Malaria

Malaria, a life-threatening infectious disease, spreads rapidly via parasites. Malaria prevention is more effective and efficient than treatment. However, the existing surveillance systems used to prevent malaria are inadequate, especially in areas with limited or no access to medical resources. In this paper, in order to monitor the spreading of malaria, we develop an intelligent surveillance system based on our existing algorithms. First, a visualization function and active surveillance were implemented in order to predict and categorize areas at high risk of infection. Next, socioeconomic and climatological characteristics were applied to the proposed prediction model. Then, the redundancy of the socioeconomic attribute values was reduced using the stepwise regression method to improve the accuracy of the proposed prediction model. The experimental results indicated that the proposed IASM predicted malaria outbreaks more close to the real data and with fewer variables than other models. Furthermore, the proposed model effectively identified areas at high risk of infection

Spin excitations in metallic kagome lattice FeSn and CoSn

In two-dimensional (2D) metallic kagome lattice materials, destructive interference of electronic hopping pathways around the kagome bracket can produce nearly localized electrons, and thus electronic bands that are flat in momentum space. When ferromagnetic order breaks the degeneracy of the electronic bands and splits them into the spin-up majority and spin-down minority electronic bands, quasiparticle excitations between the spin-up and spin-down flat bands should form a narrow localized spin-excitation Stoner continuum coexisting with well-defined spin waves in the long wavelengths. Here we report inelastic neutron scattering studies of spin excitations in 2D metallic Kagome lattice antiferromagnetic FeSn and paramagnetic CoSn, where angle resolved photoemission spectroscopy experiments found spin-polarized and nonpolarized flat bands, respectively, below the Fermi level. Although our initial measurements on FeSn indeed reveal well-defined spin waves extending well above 140 meV coexisting with a flat excitation at 170 meV, subsequent experiments on CoSn indicate that the flat mode actually arises mostly from hydrocarbon scattering of the CYTOP-M commonly used to glue the samples to aluminum holder. Therefore, our results established the evolution of spin excitations in FeSn and CoSn, and identified an anomalous flat mode that has been overlooked by the neutron scattering community for the past 20 years